Web Survey Bibliography
Title Evaluating Three Approaches to Statistically Adjust for Mode Effects
Author Kolenikov, S.; Kennedy, C.
Source Journal of Survey Statistics and Methodology, 2, 2, pp. 126-158
Year 2016
Database Oxford Journals
Access date 19.04.2016
Abstract A major hazard in conducting multimode surveys is the potential for mode effects to compromise the response distributions recorded. We evaluate the strengths and weaknesses of three approaches for statistically adjusting for mode effects. Under a regression modeling approach, adjustments are computed by regressing survey responses on mode, demographics, and other relevant variables. Under a multiple imputation approach, mode effects are conceptualized as a missing-data problem. The standard multiple imputation techniques such as chained equations can be used to impute the responses in additional modes. We also propose a new imputation approach based on an econometric framework of implied utilities in logistic regression modeling. We evaluate all three approaches using data from the second wave of the Portraits of American Life Survey sponsored by Rice University's Kinder Institute for Urban Research. This survey featured online and CATI interviewing with a national sample of adults, with random assignment to either CATI-only or web with CATI follow-up for nonrespondents. We develop a workflow to determine which variables require a mode effects adjustment based on standard false discovery rate multiple hypothesis testing procedures. We detected a significant mode effect on four survey outcomes after controlling for demographics and risk of type I error. The mode effects adjustments were then applied to these variables. The effects on the standard errors and point estimates are examined and discussed along with the advantages and disadvantages of each adjustment approach. The multiple imputation approach produced estimates with better apparent accuracy, as evidenced by better internal consistency of the estimates and a moderate increase in the standard errors. Unlike the regression adjustment approach, which can only produce aggregated estimates for the whole study, the multiple imputation approach can be used for disaggregated analysis with mode-adjusted estimates as well.
Access/Direct link Oxford Journals (Abstract) / (Full text)
Year of publication2014
Bibliographic typeJournal article
Web survey bibliography - Measurement (1822)
- Displaying Videos in Web Surveys: Implications for Complete Viewing and Survey Responses; 2017; Mendelson, J.; Lee Gibson, J.; Romano Bergstrom, J. C.
- Careless Response and Attrition as Sources of Bias in Online Survey Assessments of Personality Traits...; 2017; Meade, A. W.; Ward, M. K.; Alfred, C. M.; Pappalardo, G.; Stoughton, J. W.
- An experimental comparison of web-push vs. paper-only survey procedures for conducting an in-depth health...; 2017; McMaster, H. S.; LeardMann, C. A.; Speigle, S.; Dillman, D. A.
- Demographic Question Placement: Effect on Item Response Rates and Means of a Veterans Health Administration...; 2017; Teclaw, R.; Price, M.; Osatuke, K.
- Effects of Applying Multimedia and Dialogue Box to Web Survey Design; 2017; Chen, H.
- Role of online survey tools in creating temporally accurate Environmental Product Declarations (EPD)...; 2017; Ganguly, I.; Bowers, T.; Pierobon, F.; Eastin, I.
- A test of sample matching using a pseudo-web sample; 2017; Chatrchi, G., Gambino, J.
- A Partially Successful Attempt to Integrate a Web-Recruited Cohort into an Address-Based Sample; 2017; Kott, P. S., Farrelly, M., Kamyab, K.
- Nonprobability sampling as model construction; 2017; Mercer, A. W.
- Mobile Research im Kontext der digitalen Transformation; 2017; Friedrich-Freksa, M.
- Kognitives Pretesting; 2017; Neuert, C.
- Grundzüge des Datenschutzrechts und aktuelle Datenschutzprobleme in der Markt- und Sozialforschung; 2017; Schweizer, A.
- The Failure of the Polls: Lessons Learned from the 2015 UK Polling Disaster; 2017; Sturgis, P.
- Overview: Online Surveys; 2017; Vehovar, V.; Lozar Manfreda, K.
- Three Methods for Occupation Coding Based on Statistical Learning; 2017; Geweon, H.; Schonlau, L.; Blohum, M.; Steiner, St.
- Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 2016; 2016
- Using Visual Analogue Scales in eHealth: Non-Response Effects in a Lifestyle Intervention; 2016; Kuhlmann, T.; Reips, U.-D.; Wienert, J.; Lippke, S.
- Standard Definitions Final Dispositions of Case Codes and Outcome Rates for Surveys; 2016
- FocusVision 2015 Annual MR Technology Report; 2016; Macer, T., Wilson, S.
- Last Year Your Answer Was … The Impact of Dependent Interviewing Wording and Survey Factors on...; 2016; Al Baghal, T.
- The Effects of a Delayed Incentive on Response Rates, Response Mode, Data Quality, and Sample Bias in...; 2016; McGonagle, K., Freedman, V. A.
- Can Student Populations in Developing Countries Be Reached by Online Surveys? The Case of the National...; 2016; Langer, A., Meuleman, B., Oshodi, A.-G. T., Schroyens, M.
- The Effects of Vignette Placement on Attitudes Toward Supporting Family Members; 2016; Lau, C. Q., Seltzer, J. A., Bianchi, S. M.
- Comparisons of Online Recruitment Strategies for Convenience Samples: Craigslist, Google AdWords, Facebook...; 2016; Antoun, C., Zhang, C., Conrad, F. G., Schober, M. F.
- Comparing Cognitive Interviewing and Online Probing: Do They Find Similar Results?; 2016; Meitinger, K., Behr, D.
- Device Effects - How different screen sizes affect answers in online surveys; 2016; Fisher, B.; Bernet, F.
- A look into the challenges of mixed-mode surveys; 2016; Klausch, L. T.
- The use of online social networks as a promotional tool for self-administered internet surveys; 2016; de Rada, V. D.; Arino, L. V. C; Blasco, M. G
- Investigating Cognitive Effort of Response Formats in Web Surveys using Paradata ; 2016; Hoehne, J. K.; Schlosser, S.; Krebs, D.
- Conducting Survey Experiments Using an Online Labor Market ; 2016; Fowler, S.; Willis, G. B.; Moser, R. P.; Townsend, R. L. M.; Maitland, A.; Sun, H.; Ferrer, R.; Berrigan...
- Mode Effects on Subjective Well-being Research: Do they Affect Regression Coefficients? ; 2016; Sanchez Tome, R.; Roberts, C.; Staehli, M. E.; Joye, D.
- Examining Trends in the Presence of Survey Mode Effects ; 2016; Hisako Kitada, H.; Lesser, V. M.
- Using a Response Propensity Model to Allocate Non-c ontingent Incentives in a Web Panel ; 2016; Masterton, M.
- Retrospective Measurement of Students’ Extracurricular Activities with a Self-administered Calendar...; 2016; Furthmueller, P.
- Privacy Concerns in Responses to Sensitive Questions. A Survey Experiment on the Influence of Numeric...; 2016; Bader, F., Bauer, J., Kroher, M., Riordan, P.
- Ballpoint Pens as Incentives with Mail Questionnaires – Results of a Survey Experiment; 2016; Heise, M.
- Non-Observation Bias in an Address-Register-Based CATI/CAPI Mixed Mode Survey; 2016; Lipps, O.
- Bees to Honey or Flies to Manure? How the Usual Subject Recruitment Exacerbates the Shortcomings of...; 2016; Snell, S. A., Hillygus, D. S.
- Thinking Inside the Box Visual Design of the Response Box Affects Creative Divergent Thinking in an...; 2016; Mohr, A. H.; Sell, A.; Lindsay, T.
- Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant...; 2016; Huang, J. L.; Bowling, N. A.; Liu, Me.; Li, Yu.
- Detecting careless respondents in web-based questionnaires: Which method to use?; 2016; Niesen, A. S. M.; Meijer, R. R.; Tendeiro, J. N.
- Web surveys for offline rural communities ; 2016; Gichohi, B. W.
- On-line life history calendar and sensitive topics: A pilot study; 2016; Morselli, D.; Berchtold, A.; Granell, J.-C. S.; Berchtold, And.
- Does survey mode matter for studying electoral behaviour? Evidence from the 2009 German Longitudinal...; 2016; Bytzek, E.; Bieber, I. E.
- The impact of visual design and response formats on data quality in a web survey of MOOC students; 2016; Maloshonok, N.; Terentev, E.
- An experiment comparing grids and item-by-item formats in web surveys completed through PCs and smartphones...; 2016; Revilla, M.; Toninelli, D.; Ochoa, C.
- Establishing the accuracy of online panels for survey research; 2016; Bruggen, E.; van den Brakel, J.; Krosnick, J. A.
- Eye-tracking Social Desirability Bias; 2016; Kaminska, O.; Foulsham, T.
- Evaluating Three Approaches to Statistically Adjust for Mode Effects; 2016; Kolenikov, S.; Kennedy, C.
- Distractions: The Incidence and Consequences of Interruptions for Survey Respondents ; 2016; Ansolabehere, S.; Schaffner, B. F.